Thermal imaging camera for taking thermographic images

ABSTRACT

In a thermal imaging camera for acquisition of thermographic images of a measurement object, an electronic evaluation unit is integrated into the thermal imaging camera; it is designed for recognition of corresponding partial regions of the acquired thermographic images, and with it, the acquired images can be assembled into an overall image by overlapping the corresponding partial regions and displayed. The acquisition of the images preferably takes place during the swiveling of the thermal imaging camera over the solid angle region of the desired overall image.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.12/634,759, filed on Dec. 10, 2009, now allowed, and which claims thebenefit of priority of European Patent Application No. 09 015 237.2,filed Dec. 9, 2009, and the benefit of U.S. Provisional Application No.61/122,142, filed Dec. 12, 2008, the entire contents of which areincorporated herein by reference.

TECHNICAL FIELD

The invention concerns a thermal imaging camera for taking thermographicimages of a measurement object in the infrared region with an indicatortool built in or on the housing for a representation of the takenthermographic images, where the thermal imaging camera is made as ahand-held unit. The representation of the thermographic images may useany kind of color mapping, e.g. false color representation, grey tonerepresentation, and the like.

The invention additionally concerns a method for generating athermographic image of a large measurement object.

DESCRIPTION OF THE PRIOR ART

A system for dual band acquisition of infrared images in which the IRimages acquired in two spectral bands are combined into an image byoverlapping by means of image processing in a computer system is knownfrom the publication of M. Muller et al.: “Real-time image processingand fusion for a new high-speed dual-band infrared camera.”

A digital photo apparatus for taking images in the visible spectralregion in which an objective head is designed so that it can rotate inmanual or automatic mode on the housing is known from DE 203 05 457 U1.

A digital camera for taking images in the visible spectral region inwhich a one-dimensional panoramic image can be generated is known fromUS 2005/0206743 A1.

Thermal imaging cameras are frequently used to take thermographicimages, thus images optically acquired in the infrared spectral region,in which temperature information obtained from the images taken ofstructures or parts of structures is represented in a false colorrepresentation, thus by means of a false color scale, or by brightnessscales, or grey scales. From the thermographic images it is possible toobtain, among other things, information about the building's condition,such as condition of insulation, water damage and/or mold damage, andalso for surveillance of room areas and for monitoring or control ofmanufacturing processes.

In taking images of buildings, it is required, on the one hand, to takean overall image and, on the other hand, to take individual images ofportions of the building that are as detailed as possible. This produceshigh demands on the maximum possible resolution of the sensor field ofthe thermal imaging camera and, if the thermal imaging camera has anintegrated display tool, on the maximum possible resolution of saiddisplay tool.

Since a high maximum possible resolution gives rise to highmanufacturing costs, especially, for the detector field of the thermalimaging camera, one makes due by, optionally with a wide-angle lens,first taking an overall picture and then taking the additionalindividual pictures of the parts of the building that are of interestusing a telephoto lens, where the individual pictures offer higherresolution because of the smaller section that is chosen. Preferably theposition of these individual pictures is entered into a printout of theoverall picture for a better subsequent evaluation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B show a thermal imaging camera formed in accordance withthe subject invention;

FIG. 2 shows an object which may be at least partially imaged by athermal imaging camera formed in accordance with the subject invention;

FIG. 3 shows overlapped acquired images;

FIG. 4 shows a flow chart of a method useable with the subjectinvention;

FIG. 5 shows a stitched overall image which may be produced by a thermalimaging camera formed in accordance with the subject invention;

FIG. 6 shows load distortion stripes from image streams used to improvethe overall look and feel of a stitched image;

FIG. 7 shows a scene with objects in different distances; and,

FIG. 8 is a schematic cross-sectional view of a thermal imaging cameraformed in accordance with the subject invention.

DETAILED DESCRIPTION

The invention is based on the task of creating a thermal imaging camerathat enables the acquisition and display of large solid angle regionswith high resolution. FIGS. 1A and 1B provide an example of a thermalimaging camera 30 for use with the invention provided herein.

To solve this problem it is provided that an electronic evaluation unit80 is integrated into the thermal imaging camera 30 of the kindmentioned at the start, with which at least an acquired thermographicimage 32 can be stitched with another acquired thermographic image 32 ora stored thermographic image in the edge regions, where the images eachrepeat the same section of the measurement object in said edge region,to form a new thermographic overall image and can be provided forcombined display with the display tool 34, where the display tool 34 hasa zoom function 36 for free selection of a segment of the overall imagethat is to be enlarged. Thus, the measurement object specifies a solidangle region for an overall image, which is minimally divided intopartial solid angle regions, where each partial solid angle regioncorresponds with sufficient overlap to an image to be acquired, and theacquired images 32 are combined into the overall image at theoverlappings where they have a corresponding content. It is advantageousin this case that with each additionally acquired image 32, a newoverall image can be generated in the thermal imaging camera and theresolution of said image is not limited by the size of the overallimage. Thus, a thermal imaging camera is available with which largeareas, for example, a building or an industrial plant or a landscape,can be photographed with high resolution, and the focal plane arrayneeds only low resolution, so the manufacturing costs of the thermalimaging camera 30 are considerably reduced. FIG. 2 provides an exampleof an overall image 50 of the sensor fields taken with the thermalimaging camera 30. The display tool 34 also needs not have to have highrequirements with regard to maximum possible resolution of thepresentation if the display tool 34 is provided with controls forscrolling and/or zooming 36 (e.g., buttons, joysticks, touch pads) thedisplayed image.

The thermal imaging camera 30 that has been developed in this way isthus especially suitable for the requirements of building thermography.In particular, a thermal imaging camera 30 that can be used in one-handoperation because of its compact size, for example, at constructionsites, is made available with the invention.

Preferably, the evaluation unit 80 contains an image recognition tool,with which partial areas with corresponding content can be identifiedwhen there is a number of acquired images 32. In FIG. 8, an imagerecognition tool is integrated into electronic evaluation unit 80.

FIG. 2 shows an object 49 to be at least partially imaged. FIG. 3 (52)shows overlapped acquired images 32 (32 a, 32 b, 32 c, 32 d, 32 e) takenof portions of the object 49 identified by the image recognition tool,which may be displayed on the display tool 34. The invention takesadvantage of the finding that the computational capacity necessary foridentification of the corresponding partial regions at which the imagesare stitched together can be integrated into the thermal imaging camera30. Additional external computers, or the like, for subsequentprocessing of acquired images 32 with the goal of generating a stitchedoverall image 50 (FIG. 5) thus can be omitted. The acquired images 32are designated as images 1-20, although, as will be appreciated by thoseskilled in the art, any number of the images 32 may be utilized.

For example, it can be intended that points, lines, edges and/orstructures marked with the image recognition tool can be identified.These identified marked points, lines, edges and/or structures do notnecessarily correspond to points, lines, edges and/or structures of animage obtained in the visible spectral range and can be used foridentification of the corresponding partial regions. More generally, theimage recognition tool may be constructed such that it allows toidentify image features such as SURF, SIFT or other transformationinvariant features that may not be recognizable to the bare eye.

An advantageous embodiment of the invention can provide that partialregions in different images with corresponding content (i.e., identicalor substantially similar content) in the horizontal and/or vertical edgeregions of the thermographic images can be identified. In this manner,same image regions between different of the acquired images 32 may beidentified. It is advantageous in this case that not only a panoramicimage, as is well known, for example, in landscape photography, but alsoan image composed in two dimensions, width and height, can be generated,as is advantageous for the photography of buildings and industrialplants and the like.

For identification of corresponding partial regions in the acquiredimages 32, it can be provided that the evaluation unit 80 has a tool forpattern recognition. In FIG. 8, a pattern recognition tool is integratedinto electronic evaluation unit 80.

For convenient taking of the overall image 50 from the individualacquired images 32, it can be provided that a triggering tool 40 ismade, through the actuation of which an additional thermographic imagecan be taken. For use, thus, the camera is merely aimed at the lastsegment of the overall region to be photographed and a new image can beproduced by means of the triggering tool 40 and the stored, and combinedwith the already acquired images 32. Preferably it is provided that thedisplay tool 34 of the thermal imaging camera 30 show a timewisecontinuous image taken in the visible and/or infrared spectral regions.In this way, a swiveling of the thermal imaging camera 30 to a newposition can be monitored in the display tool 34 when using the camera30.

For an optimized display of the image currently being taken and theoverall image 50 that has been generated, it can be provided that theenlargement scale of the display is automatically matched to the size ofthe current overall image 50.

An especially easily portable thermal imaging camera 30 results when thedisplay can simultaneously show the thermographic image acquired in theinfrared region and an image acquired in the visible spectral region. Adisplay of this kind can take place, for example, by overlapping theimages, or by separate presentation in separate windows, or separatedisplays.

Preferably, however, it is provided that the thermographic image and theimage taken in the visible spectral region can be represented on top ofone another and/or overlapping and/or at least partially covering oneanother and/or fused using α-blending (alpha-blending), so thatcorresponding image segments from the same parts of the measurementobject can be represented at the same site of the display tool 34 and/orin the same display scale. FIG. 3 provides an example of an IR image 52with five acquired images 32 a, 32 b, 32 c, 32 d, 32 e, as identified bythe image recognition tool, overlapping.

A further improved possibility for generating a stitched image from theindividually taken images results when the boundary of an already takenthermographic image, preferably the last acquired thermographic image32, and/or a thermographic image currently being acquired, can berepresented on the display by means of a labeling. Preferably, thelabeling can be switched on and off. Through the labeling, a region ofoverlap can be made recognizable when taken the individual images, forexample, by the edge labeling of the acquired image 32 being moved onthe display tool 34 when the thermal imaging camera 30 is shifted into anew image-taking position.

According to one embodiment of the invention, the segment of the objectphotographed is preselected by the user and shown on the display as ablack frame.

After aiming the camera 30 at the beginning of the scene to be stitchedand starting the stitching process, the user swivels the camera 30 infront of the scene and from the frame sequence subsequent frames arestitched together and shown on the display 34 while the camera 30 isswiveled horizontally and/or vertically similar to the brush tool in animage processing program. FIG. 5 illustrates an image withbrush-tool-like stitching of the boxes 1-20.

In a further improved realization, low distortion stripes 72, 74 fromthe image streams are used to improve the overall look and feel of thestitched image, as shown in FIG. 6 (70).

Thermal imaging systems/cameras 30 tend to be afflicted with opticaldistortions that are mostly caused by the optical lenses. Due to theseso called barrel or pincushion distortions, the process of stitchingacquired images 32 together to obtain one overall image 50 getsprofoundly more complex and computationally expensive. To provide goodstitching results it is of essence to remove this distortion field andtherefore correct the geometry of the acquired image(s) 32. Usingun-corrected images results in loss of detail and accuracy.

This problem scales with the number of acquired images 32 that are goingto be stitched together and therefore provides an unwanted obstacle to avideo stitching approach where the imaging camera 30 is swiveled overthe scene. This distortion correction has to be done in real-time andtherefore, increases the overall costs of the computing unit due to itscomplexity.

The present invention provides a way to avoid this time consumingprocess and present a real-time video stitching approach for infraredimaging camera devices that is highly scalable and computationallyfrugal. Our approach takes into account, that the most commondistortions that are caused by the shortcomings of low priced opticalsystems increase their effect on the image depending on the pixeldistance of the center of the optical system. This means that there areregions in the acquired images with almost no distortion that couldintervene in the stitching process.

The approach of the present invention identifies these regions, eithermanually or automatically and provides an area where information for thestitching process can be taken without the need to apply a distortioncorrection to the overall image 50. Having identified this sub-region ofthe overall image 50 the motion between two or more consecutive framesis determined. The motion determination process is either performed bycorrelation (cross correlation, normalized cross correlation(CCF/NCCF)), motion filters (Kalman-Filter, etc.), feature-based (SIFT,SURF, LOG, etc.) approaches or motion sensors that provide the neededaccuracy. Having determined the actual vector that describes the motionof consecutive frames the new image parts can be determined and added tothe overall image 50. Because the imaging system takes many acquiredimages 32 per second and the swiveling motion of the user is typicallysmall according to the number of frames per second, the movement fromone to another pixel is quite small so that there will be a tightinformation cloud extracted out of the frame sequence that improves theresult of the overall image 50 even more. Considering the consecutiveframes do have many overlapping regions, the information extracted maybe used in the stitching process to obtain the overall image 50 with atemporal noise reduction that is simply done by weighting the additionof the frame parts that are transferred into the overall image 50. Thisprocess of intelligent combination of the partial images not onlyresults in a comprehensive overall image 50 with all information of theacquired frame sequence in it but also enhances the signal to noiseratio by filtering noise in the temporal domain.

Swiveling the imaging device over the scene to acquire a frame sequencewith which the overall image 50 will be computed one gathers a newproblem class that's concerning the image acquisition at differentviewpoints. Images taken at different viewpoints from scenes wherevisible objects differ in their distance to the spectator (imagingdevice) show deformations in consecutive frames that can't be modeled bysimple linear deformation models, such as translation and rotation. Bycombining such images, the overlap area may not include enoughcomparable information to compute an overall image 50 with no artifacts.

Our “stripe approach” that segments the image into better comparableparts is robust against such deformations evolving from scenes withobjects distrusted at different depths inside the observed scene.Because the considered stripes of consecutive frames of an imagesequence do not differ much and therefore have almost the same viewpoints, artifacts will not be visible to the human eye, other than bystitching hole consecutive image frames together. This way an overallimage 50 can be obtained which conserves the authentic look and feel ofthe scene from which it was obtained.

FIG. 7 show the result of the application of the embodiment describedabove to a 3D scene 80 with objects in different distances from thecamera 30. As one may see, the stitching causes distortions to theimage, but provides a mechanism that works well for limiting thedistortion using the almost undistorted stripes 72, 74. Therefore, the3D scene is created using acquired images 32.

Preferably, it is provided that the sensor signals of the motion sensorcan be detected and evaluated by the evaluation unit.

In one embodiment of the invention, it can be provided that theresolution of the detector field of the thermal imaging camera 30 islower than the resolution of the display tool 34. Thus thermographicimages of high quality, especially resolution, can be made and displayedwith the thermal imaging camera 30 without having to use a detectorfield with high resolution, thus a high pixel count per unit area. Sucha detector field is especially undesirable because of its highmanufacturing expense and price.

To solve this problem it is provided in a method of the kind mentionedat the start that thermographic images of parts of the measurementobject are taken with a thermal imaging camera 30, that in the thermalimaging camera 30 partial regions with corresponding contents areidentified in the acquired thermographic images 32, and that theacquired thermographic images 32 are stitched at the identified partialregions into an overall image 50 in the thermal imaging camera 30. FIG.4 provides an example method 60.

Preferably, it is provided that the assembled overall image 50 isrepresented on a display tool 34 in the thermal imaging camera 30.

According to one embodiment of the invention, it can be provided thatthe represented overall image 50 is made appropriately current whentaking an additional thermographic image by adding the additionallytaken image to it. It is advantageous in this case that the generatedoverall image 50 can be immediately checked and that individual imagesthat are still missing in the overall image 50 or that wereinsufficiently acquired can be immediately acquired after that.

To support the identification of corresponding partial regions, it canbe provided that the thermal imaging camera 30 has a motion sensor (notshown) and that the measurement signals of the motion sensor are used bythe evaluation unit to identify partial regions with correspondingcontents. It is advantageous here that the thermal imaging camera 30can, due to the sensor signals of at least one of one motion sensor(there is preferably at least one sensor provided for each independentdirection of motion of the thermal imaging camera 30) make an assessmentof the point at which corresponding partial regions could arise in thecase of successively taken individual images. This can savecomputational capacity, which simplifies the instruction of the thermalimaging camera 30.

Using the method 60 of FIG. 4, an especially easy possibility ofgenerating the overall image 50 can provide that the acquisition of thethermographic images 32 in steps 62 and 66 and/or the formation of theoverall image 50 on the display tool takes place in real time during aswivel motion of the thermal imaging camera 30 in step 64.

For dynamic adjustment to partial regions of the overall image 50 thatare of interest (and not of interest), it can be provided in step 68that the resolution can be set for each acquisition of the thermographicimages 32 in step 70. In this way, storage capacity and computationalcapacity in the thermal imaging camera 30 can be saved in step 76.Preferably, a tool for selecting the resolution, for example, a zoomfunction 36, as provided in step 74, for the currently acquired image 32can be built into the thermal imaging camera 30.

To make use of the full resolution of the thermographic image thatresults from the assembly of individual images, it can be provided thata freely selectable image segment of the overall image 50 is enlarged onthe display tool 34 of the thermal imaging camera 30 with a zoomfunction 36.

To carry out the method, the necessary tools and elements are built intoa thermal imaging camera 30. For example, it can be provided that athermal imaging camera 30 in accordance with the invention is used tocarry out the method.

An electronic evaluation unit, provided in step 72, is integrated intothe thermal imaging camera 30 for taking thermographic images of ameasurement object; it is designed to recognize overlapping partialregions 32 a, 32 b, 32 c, 32 d, 32 d of acquired thermographic images32, and with it, the acquired images 32 can be assembled into an overallimage 50 and displayed by overlapping corresponding partial regions. Theacquisition of the image 32 takes place preferably during the swivelingof the thermal imaging camera 30 over the solid angle region of thedesired overall image 50.

The invention claimed is:
 1. A thermal imaging camera for takingthermographic images of a measurement object in the infrared region,with a display tool built in or on the housing for a representation ofthe acquired thermographic images, where the thermal imaging camera ismade as a hand-held unit configured to acquire thermographic images witha swiveling motion of the thermographic camera, said thermal imagingcamera comprising an electronic evaluation unit configured to stitch atleast one acquired thermographic image with another thermographic image,where the images each repeat a same image region of the measurementobject, said thermographic images being stitched about said same imageregions to form a thermographic overall image that can be displayed withthe display tool, with correction being made for the swiveling motion ofthe thermal imaging camera by assembling the acquired thermographicimages using a set of undistorted stripes configured to limit distortionof the acquired thermographic images, where the display tool has a zoomfunction for free selection of an image segment of the overall imagethat is to be magnified.
 2. A thermal imaging camera as in claim 1,characterized by the fact that the evaluation unit contains an imagerecognition tool with which partial regions with the same image regionscan be identified when there is a number of acquired thermographicimages.
 3. A thermal imaging camera as in claim 2, characterized by thefact that the same image region may extend horizontally.
 4. A thermalimaging camera as in claim 3, characterized by the fact that the sameimage region may extend vertically.
 5. A thermal imaging camera as inclaim 2, characterized by the fact that the same image region may extendvertically.
 6. A thermal imaging camera as in claim 1, characterized bythe fact that the evaluation unit has a tool for pattern and/or featurerecognition.
 7. A thermal imaging camera as in claim 1, characterized bythe fact that the thermal imaging camera includes a trigger configuredsuch that through the actuation of which a thermographic image can betaken.
 8. A thermal imaging camera as in claim 1, characterized by thefact that the scale of magnification of the display tool isautomatically adjusted to the size of the current overall image.
 9. Athermal imaging camera as in claim 1, characterized by the fact that arepresentation of the thermographic image acquired in the infraredspectral region and an image acquired in the visible spectral region canbe represented at the same time by the display tool.
 10. A thermalimaging camera as in claim 9, characterized by the fact that thethermographic image and the image acquired in the visible spectralregion can be represented at least partially covering one another and/orα-blended, so that image segments of the same parts of the measurementobject that correspond to each other can be represented at the samepoint on the display tool and/or in the same display scale.
 11. Athermal imaging camera as in claim 1, characterized by the fact that theboundary of an already acquired thermographic image, and a currentlyacquirable thermographic image can be represented on the display tool bymeans of a switchable labeling.
 12. A thermal imaging camera as in claim1, characterized by the fact that the segment of the measurement objectacquired in the visible spectral region is larger than the segment ofthe measurement object acquired with an individual and/or with theassembled thermographic image.
 13. A thermal imaging camera as in claim1, characterized by the fact that the acquisition of the thermographicimages takes place by means of an uncooled sensor field.
 14. A thermalimaging camera as in claim 1, characterized by the fact that at leastone motion sensor, with which the swiveling motion of the thermalimaging camera can be detected, is integrated into the thermal imagingcamera.
 15. A thermal imaging camera as in claim 14, characterized bythe fact that the sensor signals of the motion sensor can be detectedand evaluated by the evaluation unit.
 16. A thermal imaging camera as inclaim 1, characterized by the fact that the resolution of the detectorfield of the thermal imaging camera is smaller than the resolution ofthe display tool.
 17. A thermal imaging camera as in claim 1,characterized by the fact that the acquisition of the thermographicimages and the construction of the overall image takes place in realtime during the swiveling motion of the thermal imaging camera.
 18. Athermal imaging camera as in claim 1, characterized by the fact that theoverall image is appropriately updated when an additional thermographicimage is acquired by adding the additionally acquired image.
 19. Athermal imaging camera as in claim 1, characterized by the fact that theresolution of the thermographic camera can be preset for eachacquisition of thermographic image.
 20. A thermal imaging camera as inclaim 1, wherein the evaluation unit is configured to identify regionsof the acquired thermographic images requiring correction for opticaldistortion.